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CHAPTER 1 



SUMMARY REPORT ON THE ORGANISATION AMD ACTIVITIES 
OF THE WORKSHOP 

Following Rosolution 1.355(d), adopted by the Qonerel 
Conference of Unesoo at its Fifteenth Session (Paris, 1968), 
a Workshop on tho Evaluation of Functional Literacy Projects 
was held at the University of London Inatituto of Education 
from the 3rd to the 21st of August, 19&9> 

The objective of the Y/orkehop, as stated in the letter of 
invitetion to participants, was :l to enable people engaged in 
the evaluation of functional litoraoy and similar programmes 
to exchange their views and experience end to oomo to conclu- 
sions as to roalistic end practical toohniouos of evaluation 
which can bo applied in their own countries." All participants 
were expected to hove a working knowledge of English. 

Invitations were sent by Unesoo to the governments of 
Member States in whioh functional literacy progrexiaos wore 
known to be operating, including those countrios participating 
in tho World Experimental Literooy Programme, asking them to 
no'..ineto participants for the Ytorkshop. In the ovont, Ecuador, 
Ethiopia, India, Indonesia, Iran, Nigeria, tho Sudon, Tcnsenia, 
Thailand, the United Arab Republic and Vonoeuola put forward 
participants. A completo list of participants is providod in 
Appendix A. 

The Workshop wot financed by Unosoo end its direction wss 
entrusted to Mr. J, B, Bowers, Adviser on Adult Litoraoy at 
tho Department of Education in Tropioal Aroos, University of 
London Institute of Eduoation. Fourtoon specialists in 
evaluation and related fields from Unesoo ond FAO and from 
Universities and othor Organisations in the Unitod Kingdom, 
whose nenos are also listed in Appondix A, served os full- 
time staff or visiting leoturors. 

A small planning group of three staff members proporod 
the Workshop programme whioh is ottnohod as Appendix B. This 
programuo was intended to be sufficiently floxlblo to allow 
adjustments to bo mode os tho Workshop dovolwpod and tho needs 
and wishes of partiolponts beooao dear. However, apart from 
small ohangos in the time-table, it was in tho event olosoly 
adhered to. 

On tho first day of tho Workshop, the participants elootod 
a Stooring and Evaluation Oomaittoo whioh was charged with the 
task of assisting on the organisation of tho programme ond 
oppreising its achievements. Tho Committee met regularly 
throughout the Workshop and providod o very valuoble sorvloe 
to tho participants and staff. 



The program© maintained • bslnnoe between meetings with 
in the Institute of Education and visits to organisations and 
institutions in and near London* 

The meetings in the Institute were themselves varied in 
purpose and format# During the first weoki statements by 
participants on their partioular problems end experiences in 
evaluation were alternated with contributions from visiting 
specialists or staff members. These oovered the following* 
subjeotst 

Functional Literaoy *•- Definition and Evaluation, 
by Mr, j, B, Bowers} ' 

The Evaluation of Functional Liters oyt en Economist* s 
View, 

by Profossor M, Blaug} 

The Bole of Cost Analysis in the Evaluation of 
Functional Literaoy, 
by Dr. B. Jolly | 

Techniques of Sooial Study for Evaluation, 
by Dr. B. Wiener) 

The Evaluation of Kural Extension Programmes, 
by Dr, J. oi Franco) 

Experiences and Problems in the Evaluation of Functional 
Literaoy i a Sociologist *8 View, 
by Dr. K. Hoys. 

These lootures were followed by dlsousslons of the prob> 
lens raised. 



Early in the seoond week the Workshop broke into three 
working groups deoling rospeotivoly with l 

Polioy. Planning end Organisation in the Evaluation of 
Funotionol Literaoy) 

Methods end Toohniques used in the Evaluation of Funoi. 
tionel Literaoy) 

Cost/Bonofit Analysis in the Evaluation of Funotionol. 
Lltoracy. 



the 

tho 



On the third dey after the etert of their deliberations, 
working group* node interim reports of thoir progress to 
wholo Workshop* Thoir find reports woro presented during 
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the lest week. 

Hr. J» C. Cairns, Director of the Literaoy Division in 
the Department of Out-of-School Education of Unesoo visited 
the Workshop for two days at the beginning of the second week 
end held en informal but most useful and informative dis- 
cussion with the staff and partioipants on problems of 
evaluetion in Unesoo's Experimental World Literacy Programme. 

Also during the second week, visits wero made to: 

the Testing and Development Unit of tho City end Guilds 
of London Institute, en organisation which is actively 
oo-cpereting with industries and industrial training 
boards in providing training programmes and testing 
procedures for ell levels of technical and vocational 
training | 

the National Coal Board in London which provided a 
detailed description of training and retraining 
programmes in tho mining industry and their o-oluation; 

Pressed Steel Fisher Ltd., at Cowloy, noar Oxford, to 
learn about evaluation ond training procedures in the 
motor Indus try | 

the Overseas Visual Aids Centre (OVAC). and the Centre 
for Eduoationol Television Overseos (CETO) to discuss 
the relationship of action research ond evaluation to 
the preparation of medio for functional literaoy. 

A number of oulturol visits were olso oado to places of 
Interest in and around London ond Oxford. 

Towards the end of tho Workshop a full day was dovotod to 
the dotailed study of the Unesco Manual on the Evaluation of 
Experimental Literaoy Projeots. This was introduoed by a 
most helpful talk from the author of the Monual, Dr. H. P. 
Sakseno, Head of tho Literaoy Evaluation Unit in the Un06co 
Deportment of Out-of-School Bduootion. 

A norrotive format has been ohosen for the Report on tho 
Workshop, aotivitios being described noinly in the tlmo 
sequence in whioh they occurred. Zt has not, of oourso, beon 
pmssiblo to give c detailed account of oil the Workshop's 
activities. Tho intontion rather is to presont a synthesis 
of tho contributions mode by individual partioipants, des- 
cribing their problems ond oxporionces, a summery of 
the leotures given by visiting specialists ond of tho dis- 
cussions whioh oroso from those contributions. These ore 
supplemented by tho reports of the throo working groups, ' 



accounts of educotionol visits ond o brief rsoord of disoussion 
on the Unesoo Monad. (1) 

An analysis of the main ideas emorging from the Workshop 
follows in Chapter 2* 



(1) tfnesco Me nuel on Adult end Youth Education* Sveluation of 
Experimental Lite rosy Projects* - Provisional version 
document ED/V8A35 Paris, 85 pp. 
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CHAPTER 2 

IDEAS AND SUGGESTIONS PUT FORWARD BY THE WORKSHOP 
Definitions 

The Workshop began by discussing the meaning of a number 
of current terms. ’Functional literacy’ (as distinct from 
’simple literacy’) was taken to mean a comprehensive programme 
of education and treining for illiterote (and sometimes ceoi- 
literete) adults* with a literaoy component built in. 

Agreement was also reaohod on the moaning of ’work-oriented 
literacy’ i the ’selective-intensive strategy* os applied in 
the Experimental World Literooy Programme* ond * development ’ 
and ’modernisation’, which were regarded os essential aims of 
functional literacy* These torms are further discussed in the 
next chapter. The Workshop oooeptod the definition of ’ovoluo 
tion* used in the Unesoo Manual I ’'evaluation olms ot norsuring 
whorever possible in quantitative terms, ond oooording to 
well-defined oritoric, tho major direct and indiroot of foots 
of a oortoin octivity, toking into oooount its objootivos as 
ostoblishod before its inooption*" A distinction was drawn 
botweon ovoluntion in this sonso ond ’action-oriented re scorch 
or ’action rosooroh* , which was dofinod cs "systomatio study 
and investigation, tho results of which arc fed back dirootly 
and imnodiatoly to tho oporotionol staff to help them to 
improve their day-to-day work." 

Aims of ovrluation in functional litoraoy projeots 

An important purpose of evaluation is to provido tho 
financiers, plcnnora and administrators, who clloooto funds 
for functional litoraoy, with objeotivo information on how 
theso funds hovo boon spont. Rolatod to this is tho nood to 
shed light on tho inter-rolntionship botweon litoraoy and 
developaont ond to study its significance, in ordor to detor- 
mino the plaoo and priority to bo givon to litornoy in 
development plans t hones olso the importance of oost-bonefit 
analysis* Evaluation should olso holp to onswor tho question! 
"what sort of litoraoy?" Thus it should study tho offeots of 
different mothods and media* For exonplo importance was 
ottoohod to the oonparotivo ovrluation, in torms of oost- 
effeotiveness, of work-orientod functional litornoy in which 
litoraoy toaohing is integrated into vocational training, as 
ogainst simple literaoy lending on to soporoto vocational 
training* Tho offioienoy of different types of organisation 
should also be analysed* 



Evaluation and lotion Reaearoh 



In oontraat to this long-range purpose of evaluation ite . 
more immediate purpose ia to help the operational staff with* 
in eaoh projeot - to aot as their 'controlling dsvioe', and 
even as their 'defenoe meohanism* against over-ambitious 
expectations from outside* The first is 'external' the 
aeoond 'internal' to the programme* The first is particularly 
relevant to the Experimental World Literaoy Programme, the 
aeoond to the euooess of the looal projeot* The staff of 
projeots within the World Literaoy Programme must therefore 
respond to two demands i first, to aooopt international norms 
for evaluation established by the Uneaoo Evaluation Panel, in 
the interests of world-wide comparability, end aeoond, to 
adapt these to the looal situation* They mey therefore need 
to establish both general objectives and looal operational 
objeotives. In any case, effootivo evaluation requires a 
much more precise definition of objectives - and in measurable 
terms - by projeot planners than has so far been evident, and 
evaluators must assist them in this task* 

In fulfilling its immediate purpose of helping projeot 
operators, evaluation takes on the role of aotion reaearoh and 
the relationship of evaluation and aotion reaearoh was a 
recurring theme of disoussion in the Workshop. It is olear 
that there are oertain anomalies between the long-range pur- 
pose of meosuring the offeota of funotional literaoy on 
development and the immediate purpose of helping the opera- 
tional staff to improve their on-going aotivities* If it was 
a question of ohoioe, it was agreed that priority should be 
given to long-term evaluation, at least in the Experimental 
World Lite.'aoy Projeots, and the temptation to eonoentrate 
entirely upon the less rigorous methods and more immediate 
benefits of aotion reaearoh should be resisted* There was, 
however, general support for the 'intermediate' position 
taken up by the Uneaoo Manual, namely that the evaluation 
programme should serve both long-term and short-ters, exter- 
nal and internal, purposes* This would involve three 
oonourrent and overlapping types of aetivity* 

- the measurement of the 'diroot and indlreot effects' 

of funotional llteraoyi 

- speolal studies to otaluate operational methods and 

media, whioh will provide data both of world-wide 

interest and of looal value | 

- notion reaearoh to holp planners and operational staff 

to control and improve the programme* 

The suggestion was put forward thst aotion reaearoh 



m y m 

should bo built into the experimental production of medi®^ 
and instructional systems for functional literacy, with an 
action research worker assigned to the media produotion team. 
Action research in this context would inolude studies designed 
to rolete the content of media to the needs of the target 
audience and to the educational aims of the programme, as 
wall as 'pre-testing', or 'operational tests', as they are 
called in the Unesoo Manual. This type of aotion research, 
intended to feed bock data to the media producers - to 
improve media during produotion, was distinguished from 
evaluation of the effectiveness of media once completed, 
distributed and used in the programme. 

Organisational Structure for Evaluation 

A national evaluation unit for an important functional 
literacy programme should ideally inolude a full-time 
direotor and full-time or part-time professional staff with 
both international 'and national experience in the fields of 
economics, sociology, psychology and statistics. It should 
engage short-term consultants in specialised areas and 
research workers to undertako speoial studies. 

Tho unit should havo working relations with Universities, 
and roseoroh Institutes, both inside and outside the country, 
as well as with various government departments, agencies and 
organisations whose support and assistance may be valuable. 

Each ' sub-project' in the country should hove its own 
small evaluation unit including a competent social scientist 
and appropriate full-time and part-time field staff. 

It was considered important to resolve the apparent 
conflict between the need for independence end objectivity 
in the evaluation unit end tho need for close collaboration 
between the unit and the operational staff. With this in 
mind two possible structures were envisaged. 

- Tho national evaluation unit could be located outside 
the project and be independent, in which case a 
working liaison committee, including members of the 
operational staff and the evaluation unit, should 
be set up to ensure collaboration. 



(1) The term 'media' is used here in e general sense to 
include all kinds of eduootionol 'instruments' from books 
through audio-visual olds to radio and television programmes. 
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m The notional evaluation unit might bo looated within 
the operational staff) with responsibility to the 
projeot director) in whioh ocso links should be 
ostoblished with on outside orgonisotion or organi- 
sations) suoh as o Notional Evaluation Committee or 
an appropriate University deportment or Institute) 
whioh would lay down norms ond exemino the results 
of evaluation. * 

The question wos deboted whether or not cotion research 
should bo a funotion of the evaluation unit* The working 
group dealing with Polioy) Planning and Organisation suggested 
that if the evaluation unit was a port of the operational 
staff it should be rosponsiblo for oction research) but not 
if it wos established as on independent unit* Eventually) in 
its discussion of the Unosoo Manual) the Workshop took the 
view that the throo activities (long-term evaluation, spooiol 
studios, action research) should bo handled by a single 
'Research and Evaluation Unit,' 

Polioy ond Procedures for Evaluation 

There wos general ooooptance of the evaluation procedures 
proposed in the Unosoo Manual, at least as an ideal to be 
aimed at* Those would involve the Research ond Evaluation 
Unit, during the preparatory phases in rooruiting ond training 
its staff; in the oollootion of background data, (from infor- 
mants, documentation ond observation) 5 in problem surveys 
(to detect the principal problems of the projeot aroas); in 
technical feasibility studies and operational tests (to try 
out methods and media)* It wos agreed that these activities 
were intended to provide information to the unit, os well as 
data for project planning and development* They were thus in 
the realm of notion research rather than evaluation* The 
unit should then be involved with the project planning staff 
in defining the objectives of the projeot and in selecting 
experimental ond oontrol groups, if the use of controls 
proved locally acceptable. 

Once experimental and control communities ond work-units 
(suoh as factories, mines, farms) have been chosen and the 
operational objectives of the programme and its various 
oourses defined, the evaluation unit oan establish its metho- 
dology, select what the Unesoo Manual calls 'indicators, 
elements and minimum measurements for evaluation', and begin 
its 'community-level* ond 'work-unit-lovol' base-line surveys* 
The 'participant-level* base-line survoys oannot however, 
commence until the participants are enrolled in dosses ond . 
oourses in the operational phase of the projoot. 
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Thereafter 'interim' and 'terminal 1 aurveya will be 
oarried out at the three levels , to measure , against the 
initial base-line data, the ohanges that have taken place* 
Comparison with control groups, as well as continuing 
observation through so-called 'process studies', should 
then make it possible to isolate the effeots of the 
independent variable of functional literacy* 

Speoial studies on educational methods and media,' and 
on organisation and administration could be conducted during 
the operational period and every endeavour should be made to 
analyse costs and benefits of the funotionel literacy 
programme and of epecifio methods used in it* 

Whilst accepting this general scheme of aotion research 
and evaluation the Workshop was under no illusions regarding 
the difficulties of applying it in the conditions prevailing 
in most of the projects of the Experimental World Literacy 
Programme, let alone in other functional literacy projects* 
There was general gratification that Unesco had abandoned the 
impossible aim of trying to measure the effects of functional 
literacy at the national level - on the eoonomy of a whole 
country^ The Workshop rocognised the importance of evaluation 
at the lovels of tho local community and the work unit, but 
was nore inclined to the view expressed by Professor Bleug 
that the evaluation team would have a substantial achievement 
to its credit if it could arrivo at conclusive measurements 
of the impact of such a programme on individual participants, 
plus some effects on the work unit* 

Methods of Sooial Research 

The Workshop assigned one of the working groups and a 
considerable proportion of its timo to discussion of the 
methods and techniques of social research which were appli- 
cable to evaluation and, move specifically, of how they should 
be applied. 

This study encompassed methods of sampling, seleotion 
and training of interviewers, interviewing design end inter- 
viewing, observation, sampling, selection and use of control 
groups, assessment techniques, achievement tests, sociomotrio 
techniques, intorest soalos and motivation analysis* It also 
dealt with tho special problems of comparative evaluation of 
alternative methods in functional literacy* It emphasised 
the point thct evaluation studies should concentrate upon 
behaviourol change rather than attitudinal change* An 
evaluator is not so much interested as to whether farmers 
have changod their ottitudss, for example, to education or 




It should be stated that Unesco did not set before itself the 
aim of measuring the effeots of functional literacy at the national 
level or on the economy as a whole .The misunderstanding has perhaps 
arisen because the Ouide recommended collection of some data at 
national level .The purpose of this was not to establish benchmarks 
for comparison with similar data at a later date but to get an 
understanding of the context in which the programme was introduced* 
Please see in this connection page 134 f last paragraph* 
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agricultural innovation, os in whothor they bovo ohonged their 
behaviour in these fields as o result of the functional literacy 
campaign. Quite apart from the difficulty of measuring atti- 
tudes, the assumption tbot a pore on's behaviour changes to 
matoh changes in ottitudo doos not clwcys hold up in preotioo 
ond doos not justify measuring the intormodiary rothor than 
tho ond stage of behavioural change* 

Problems met in the Evaluation of Functional Litorooy 

Early in the Workshop participants gave aooounts of 
evaluation and action rosoarch in projeots of functional 
literacy and rolotod fields, Do spit o substantial eoonomio 
ond sooiol differences between the countries studied, a wido 
range of similor problems recurred. 

Most oruoiol perhaps was the shortage- of competent research 
end ovoluation staff at all levels - international, national 
and loocl, ond the consequent problems of recruiting tho 
ovoluation unit. Related to this was the shortage of timo in 
tho one year preparatory period. In several countries, where 
vornooulor. languages woro preponderant, language policy was a 
cause of conflict ond language differences handicapped the 
reoruitment ond oporotions of tho staff. Governmental- atti- 
tudes to evaluation wore not olways helpful, and a particular 
difficulty arose in achieving co-ordination between departments 
concerned with development ond those ooncornod with literacy. 

The process of dato gathering was hondicopped by a variety 
of factors - tho absence of documentary material and relioblo 
census reports, tho physical probloms of communication in rural 
oroas, tho rosistanco of rural people to providing information 
on their economic status, ovon tho rosistanco of factory mana- 
gers to releasing data on production or industrial procosses 
which might benefit competitors. 

Finally there were the problems of defining measurable 
objectives end suitoblo indicators ond, more difficult still, 
of establishing controls, where this meant withholding tho 
benofits of the project from certain groups ond communities, 
whoso members must nevertheless submit to interviewing ond 
observation. 

Other problems woro brought up by visiting looturers. The 
economists in particular strossed the very rool difficulties 
of measuring tho impact of functional litorooy on production. 

At the level of the work-unit - factory, mine or form - this 
was an extremely complex exercise and serious doubts wore 
expressed on the possibility of isolating the effoots of 
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functional literacy from those of other factored such as 
improved seeds , fertilisers, land re form , communications | 
changes in the market- for produce and variations of olimate 
in the rural seotor and machinery, management and materials 
in an industrial setting* It was significant that in the 
comparatively sophisticated situation of British industry 
it was generally considered impossible! and perhaps even 
undesirable! to measure the effects of training as an inde- 
pendent variable - to evaluate its impact in quantitative 
terms - on production in the work-unit* 

Even at the participant level the isolation of the 
effeots of training on production was not an easy exercise* 

One possibility was to ooncontrate on the individual's 
earnings, but this would only be valid if the labour market 
worked perfectly competitively. Another approach was through 
job-analysis, but this would require that the skill-oontent 
of every job be analysed, together with the means of acquiring 
the necessary skills and whether literacy was required for 
this purpose. The aim envisaged by the Unesoo Hanual of 
measuring the contribution of the individual worker to 'the 
production or other purposes of an enterprise' was regarded 
as unattainable. 

The problems of cost-benefit analysis were also discussed , 
amongst the.n the difficulties of quantifying 'opportunity/ 
costs' and 'implicit benefits'* The possibility of arriving 
at 8 coat-beuefit ratio for functional literaoy, by encom- 
passing all benefits in a single number and contrasting this 
with six costs, again in a single number, was considered 
illusory and a warning was given against drawing simple con- 
clusions from the study of cost-effeotiveuess in isolation. 
These difficulties and dangers were not, however, regarded aa 
diminishing the importance, or precluding the possibility, of 
oost-benefit or oost-effeotiveness studies, to which the 
Workshop gave careful attention. 



Summary of Suggestions emanating from the Workshop 

Although the problems of evaluating functional literaoy 
were recognised to be many and serious there was a real deter- 
mination in the Workshop that they must be resolutely faced. 
The success of the Experimental World Literacy Programme 
depended upon it, since there oould be no value in experi- 
mentation without evaluation. 

Although the Workshop was not colled upon to givo advice 
or make recommendations, it may be useful to summarise at this 
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point some of the wore interesting idoae and suggestions 
wbioh will be reoorded in greater detail In She following 
obapters of this report* 

As a palliative to the shortogo of evaluation staff and 
a contribution to their effiolenoy it was suggested that 
Unesoo might i 

- approaoh Universities to provide integrated ooursoe 

suitable for the training of projeot evaluators | 

- bring evaluators togother in high-level oonferenoes} 

- provide systenotio training for evaluation teams in 

eaoh oountryi 

•• give sore clear-out instructions on mothodology to be 

oppliedf 

- list the speoiol studies end experiments it wishes to 

see undertaken in projeotef 

- issue a bulletin on funotionel litercoy end its 

evaluation. 

Universities should be enooureged not only to provide 
training but also to take a more active pert in resooroh and 
evaluation in funotionel literaoy projects. 

More attontion should be paid in evaluation programmes! 
and in training for them t to data processing and analysis. 

National evaluation units should provide orientotion to 
operational stoff in eaoh projeot in order to give them an 
understanding of the value t aims and requirements of evalua- 
tion, as well as training in evaluation prooedures which they 
will have to apply* In partioulor, teaohers end instructors 
should be trained in building testing techniques into 
training programmes* 

The 'preparatory period* , for a funotionel literaoy 
projeot, originally one year, should be floadble enough to 
permit the eveluation unit to establish its design and metho- 
dology, select its experimental and control groups, and to 
•tart its base-line surveys* 

The various government deportments and other agenoies 
whose collaboration would be neoessory to the evaluation, os 
well os to the operation of the projeots, should commit them- 
selves to full co-operation, if possible by signing the 
initial agreement with Unesoo* 
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The definition of general and operational objectives, in 
behavioural and measurable terms, for each project, sub-project 
each training course or extension programme - was regarded as 
a prerequisite for effective evaluation and as needing more 
serious attention than heretofore* 

Evaluation should focus primarily on the effects of func- 
tional literacy on participants and their families, seconda- 
rily on its effects in the work units and communities and not 
at all on the impossible task of measuring its effeot at tho 
levels of the region (e.g. whole projeot area) and country. 

A variety of tosts must be devised to aid the selection 
of participants and matching control groups, and to measure 
achievoment of participants at succeeding stages of coursos 
and programmes. The Workshop examined tests used in the 
training of operatives in the British cool mining and motor 
industries. A suggestion was made that Unesco should commis- 
sion a vocational training expert to prepare a manual on job 
analysis end test design* 

An important end monegeablo area of study might be the 
extent to which literacy effects the efficiency of extension 
and training programmes, and another, related to this, the 
comparative evaluation of composite work-orionted functional 
literacy as against simple literacy leading on to separate 
vocational training. These might bo fruitful areas for cost- 
effectiveness studies* Evaluation of instruotor and teacher 
training end performance will also bo a crucial moons of 
improving tho quality of tho programme. 

Evaluation ot the participant level may aim to measure 
throe kinds of change: attitudinal, (e.g* acceptance of the 
importance of fertilisers); behavioural, (e.g. correct use of 
fertilisers); economic, (e.g. increased crop production). 

Tho second is likely to bo the most useful and aonogeablo. 

Tho difficulties of measuring tho economic consequonces 
of functional literacy in terms of productivity have already 
boon mentioned; for this purposo evaluation should bo ot a 
low-enough lovol (e.g. ono production line rather than a 
whole factory) for tho ohanges measured to be attributable to 
the training given. 

Systems analysis ond cost-effectiveness studies may use- 
fully bo applied to various forms of organisation. The cost- 
of fectivenoss of various instructional systems should &lso be 
evaluated. 
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In view of tbo difficulty - end importonoo - of cost- 
benefit anolysls, pilot experiments to pro-test techniques 
night be oorriod out in one or two seleotod projects, whore . 
objectives were defined in meosurablo terms, whero explicit 
ond quontifioble benefits oould be entioipotod end whore the 
effioionoy of budgeting ond book-keeping could be relied 
upon* 



The usefulness of evoluotion must ultimately depend on 
the way its findings ore oommuniooted to deoision makers* 

For this purpose a roport is needed whioh con bo understood 
ond interprotod by the loymon - the person with little or no 
troining in the sooiol soienoes* At tho some timo socirl 
scientists will wish to hevo fuller detoils of tho design ond 
methodology used and the data obtained. Rather than two 
separate reports, the best solution was thought to be a short 
ond rondablo report with full supporting nppendioes 
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CHAPTER 3 

FUNCTIONAL LITERACY t DEFINITION AND EVALUATION 

by 

JOHN BOWERS 



The Workshop is concerned with the evaluation of functional 
literacy, so it is suggested that as little time as possible 
should be taken in disoussing functional literacy itself* 
Nevertheless one cannot realistioally discuss evaluation without 
considering what is to be evaluated* This working paper, there- 
fore, deals with definitions of 'literaoy', terms used in 
describing it, and some of the problems which arc likely to 
arise in evaluating it* 

Adult Literacy and Schooling 

Although the primary school is the most important insti- 
tution for eradicating illiteracy, the Workshop is concerned 
with the evaluation of adult literacy and related programmes 
of adult education and training* Unless otherwise stated, 
•adult' will be token to mean 15 years end over - the ago 
normally used for census purposes. 

Definitions of Literacy and Related Probloma of Evaluation 

Literacy is defined os 'the degree to which an individual 
possesses the mastery over symbols in their written form, or 
is able to, encode and decode written messages - to write and 
to rood.' (l) One of the first aims of literacy evaluation 
may be to measure how for participants in a programme havo 
achieved this* a process that may conveniently bo called 'the 
evaluation of reading and/or writing ability. ' 

For census nurnosos, Unesco has defined a literate os 'a 
person who can road with understanding ond write a simple 
message related to his everyday life*' This lovol' of attain- 
ment might suitably be described os 'elementary litorocy. ' 

This definition contains two relative terms - 'under- 
standing' end 'simple', the one relevant to the roodor, the 
other to the message* The reading ability of a reader 



(1) Rogers, Everett M, 'Modernisation emong Peasants*' 

Holt, Rinehart & Wilson, p.429» Sec Ch.4, p.?2. (Available 
for study.) 
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naturally relates to the readability (the level of difficulty) 
of the mo a so go (roading matter) to bo road. If the evaluation 
is to test tho reader's lovel of reading ability, it will bo 
nooessary to establish standards of diffioulty in tho test 
reading matter (texts) to bo road. If it is ooncerned to 
evaluate tho message, the roeding matter (e.g. in pretosting 
reading aotoriols), it will be nooessary first to establish 
tho lovel of roading ability of tho 'torgot audience' - tho 
rendors for whom it is intonded* 

This is not as simplo as it sounds, especially if it is 
desired to standardise levels of litorcoy (and tests) for 
difforont cultures and languages. 

Levels of literacy among adults are somo times equated 
with lovols supposed to be attained by ohildron after a 
stated number of years of primary sohooling, but this, for 
obvious reasons, is also rolntive and of doubtful vcluo. 

Numorccy 

Literacy is of ton regarded as inoluding some skill in 
arithmetic and even algebra and geometry (the third of tho '3 
Rs'). Evaluation may, therefore, also bo concerned with testing 
'nuaoraoy. ' 

Interpretation of Visual Images 

Another skill, whioh is complementary to literaoy and 
numeracy is mastery over visual symbols other than words and 
numbers- especially representations of 3 dimensional 
'reality* on 2 dimensional (flat) surfaces - pictures, photo- 
graphs, diagrams, maps, oharts etc. Illiterates generally 
have great difficulty in the perception and interpretation of 
visual images, and this may have to oe taught and evaluated. 

Here again we must distinguish procedures for testing the 
perception and understanding of the viewers from those 
designed to evaluate the communication aohleved by the visual 
images (e.g. . audio-visual aids.) 

Languages and Literaoy 

Literacy is essentially an extension of the use of language 
from speaking and hearing to writing and reading. VJhat language? 
In multi-lingual areas this may be a difficult question and a 
sound language policy may be an essential basis for a literaoy 
programme. There may, for example, be a conflict between the 
psychological advantages of making a person literate in his 
mother tongue and the sooial, political or other advantages of 
doing so in e language of wider communication! e.g. the 
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offioiel national language. If functional literaoy is to be 
'work oriented' (see below) the obvious choice may bo tho 
language of work or of training for work. It is, however, 
moro difficult, end tokos muoh longer,, to teach literaoy in a 
second/foreign language than in the learner's mother tongue. 

If, however, the mother tongue is on unwritten longuoge any- 
thing up to three years of work by linguists will be needed 
to analyse its dialects, give it an appropriate alphobet, 
establish its grammar, prepare word lists and perhaps a 
dictionary, before it oan be used for literaoy. 

A literaoy programme mav. therefore, have to comprise: 

a, linguistic researoh on languages spoken to determine 

longuage policy; 

b. linguistio research (o.g. word, sylloblc, letter 

frequency counts) on written languages os a basis 
for tho production of teaching materials; (l) 

c. linguistic research on unwritten languages; 

d, comparative evaluation studies of literacy teaching 

methods (e.g. in tho mother tonguo as ogainst o 
o socond language; of teaching o second language 
through the mother tongue os against diroot 
teaching of the second language; 

o. tests of linguistic ability (speoking/understanding; 
rooding/writing) in a socond longuoge. 

'Simple* Literacy versus 'Functional' Literacy 

Much of what has been written above refora to literacy 
p er so - as nn end in itself - os cn extension of the ability 
to communicate - irrespective of the purposes for whioh this 
ability is to bo used. The term 'simple' litoracy sooms 
cppropricto to distinguish this ability from 'functional' 
literaoy. A moro oommon practice is to distinguish 'olemontnry' 
literacy from ' functional' litoracy, but this is o falso 
antithesis, for thcro can be olomentcry levels of functional 
literacy; though, of course, the more advanced it is the more 
likely it is to bo ' functional . ' It is therefore suggested 
that the contrast should be drewn between 'simple* and 



(1) In tho Vale do Rio Doce literaoy project in Brazil a 
computerised study of sylloble frequency in Portuguese is 
said to have halvod tho time token to teach literacy to 
miners. A report in French is available. 
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